High-Speed Motion Tracking by Direct Access of a CMOS Sensor∗

This ICCV’03 demonstration will present a new custom CMOS camera, which has been designed for very high performance in real-time tracking applications. We make use of the direct pixel access capabilities of a non-integrating CMOS sensor (Fuga 1000) and have designed an FPGAbased solution to fixed pattern noise calibration, random subwindow access, and USB2 interface connection. This device can handle two sensors simultaneously (synchronized stereo) and can provide up to 25 individually addressed and sized windows at rates of 2kHz. We will present a tracking application which localizes and keeps track of randomly moving targets under hard real-time constraints. 1 CMOS Camera Architecture We have developed an architecture with directly addressable multi-window capability and optional synchronized stereo functionality. CMOS camera technology has important benefits for tracking, like high bandwidth and very high refresh rates referring to window size. The current design consists of several blocks pictured in figure 1(a). For both image sensors shown at the top of the diagram we use a Fillfactory FUGA 1000, a 2/3” monochrome, logarithmic, non-integrating, random-accessible CMOS area image sensor with a resolution of 1024 x 1024 pixels. The sensor features are high fill factor and high dynamic range. The non-integrating illumination behavior of this sensor demands a correction of fixed-pattern-noise (FPN). For this purpose we use large RAM blocks for ∗We gratefully acknowledge support by the Austrian Science Foundation under project FWF P15748, and by the Christian Doppler Laboratory for Automotive Measurement Research (a) Block diagram (b) Stereo configuration Figure 1: Our custom CMOS camera architecture each camera to store several calibration images and frame data as well. To handle very fast window addressing, to obtain synchronized window readout from both cameras and for FPN correction on the fly a lot of glue logic must be added. To gain efficiency in terms of speed and size the whole logic is implemented in an FPGA device. We chose a USB 2.0 interface to the host computer for high transfer rate, small size and good performance. 2 Tracking Application Having access to small subwindows which can be addressed and read out at update rates of 2kHz permits us to reliably track fast and arbitrary moving objects. We will show this capability with the following demonstrator, which is currently being assembled by our workshop: We will use a frame with dark background, transparent cover, solid side-walls, and a wire-grating top and bottom. Air-flow will be generated by strong ventilation from bottom to top. Table-tennis balls will be inserted into the frame, which will move around in a random manner. The task is to simultaneously track all balls with one CMOS camera. This demonstration is focused on the specs of our camera and its application to very fast motion tracking. Tracking is done in small subwindows, which remain centered at the individual blobs due to the 2kHz update rate. We are taking full frame images only at initialization, later on only subwindows (typically 15 × 15 pixels) are grabbed. Furthermore, the non-integrating behavior of the sensor results in low motion-blur and the logarithmic characteristic allows a wide range of illumination intensitites.